Hierarchical Modeling and Recognition of Manipulative Gesture

In order to achieve natural, proactive, and non-intrusive interaction between humans and robots, the understanding of human actions is a highly relevant task. In this paper, a vision-based method for manipulative gesture recognition is proposed. Different from the traditional trajectory-based approaches, the manipulative actions are modeled not only based on the hand trajectories but also on the object context. This context-based trajectory recognition is embedded in a hierarchical hidden Markov model which represents the hierarchical structures in manipulation tasks. For representation, a lattice dynamic Bayesian network is used and the inference is done by particle filtering. The results of experiments in an office environment show the applicability of this approach for recognizing manipulative gesture.